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DETAILED ACTION 

Response to Amendment 

1 . Applicant's request for again reconsideration of the finality of the rejection of the 
last Office action (mailed on 1/3/2008) is persuasive during the interview the Applicant 
had on 1/26/2008 with Charles Rones and the Examiner and, therefore, the finality of 
that action is withdrawn. 

2. Applicant's Amendment filed on 5/1/2007 has been entered with newly added 
claims 19-38 and cancelled claims 1-18. In this Office Action, claims 19-39 are 
pending. 

Claim Rejections - 35 USC §112 

3. The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

4. Claims 19, 28 and 38 are rejected under 35 U.S.C. 112, first paragraph, as 
failing to comply with the written description requirement. The claim(s) contains subject 
matter which was not described in the specification in such a way as to reasonably 
convey to one skilled in the relevant art that the inventor(s), at the time the application 
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was filed, had possession of the claimed invention. Applicant specification does not 
properly defined the phrases "training dataset" and it is considered as new matter. 
Applicant did not define identifying step in the specification. Therefore Applicant is 
claiming a new matter, see for example, claim 19, line 5. 

5. following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

6. Claims 19, 28 and 38 are rejected under 35 U.S.C. 112, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject matter 
which applicant regards as the invention, "identifying one or more predetermined feature 
types within each image, or a region thereof, of a plurality of images of a training 
dataset." Applicant has left several steps before the identification step. 

7. The claim 21 is rejected under 35 U.S.C. 1 1 2, second paragraph, as being 
indefinite for failing to conform to current U.S. practice. Claim 21 is rejected, because 
pronouns are not permitted in the claim. Only what is being referred to by "its" should 
be set forth in the claim. Thus, the claimed on page 3, line 2-3 recitation as "geometry 
its scale" renders the claim vague and indefinite. 

Claim Rejections - 35 USC § 101 



8. 



35 U.S.C. § 101 reads as follows: 



Application/Control Number: 10/813,201 Page 4 

Art Unit: 2164 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

9. Claims 1 9-39 are rejected under 35 U.S.C. §101, because none of the claims 
are directed to statutory subject matter. Independent claims 19, 28 and 38 merely 
claiming nonfunctional descriptive material, i.e., abstract ideas. Even when a claim that 
recites a computer that solely calculates a mathematical formula or a computer disk that 
solely stores a mathematical formula is not directed to the type of statutory subject 
matter eligible for patent protection. The claims are not producing useful, concrete and 
tangible results. See Diehr, 450 U.S. at 186 and Gottschalk v. Benson, 409 U.S. 63, 
71-72 (1972). 

Claim Rejections - 35 USC § 103 

1 0. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

This application currently names joint inventors. In considering patentability of the claims under 35 
U.S.C. 103(a), the examiner presumes that the subject matter of the various claims was commonly 
owned at the time any inventions covered therein were made absent any evidence to the contrary. 
Applicant is advised of the obligation under 37 CFR 1 .56 to point out the inventor and invention dates 
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of each claim that was not commonly owned at the time a later invention was made in order for the 
examiner to consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) prior 
art under 35 U.S.C. 103(a). 

11. Claims 19-39 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Mojsilovic et al. (US Patent 7,043,474) hereinafter Mojsilovic, in view of Jain et al. (US 
Patent 5,91 3,205) hereinafter Jain, and in view of Essafi et al. (US Patent 6,642,929) 
hereinafter Essafi. 

12. As per independent claim 19, 28 and 38, Mojsilovic teaches a method for 
characterizing, annotating and determining image similarity based on semantic meaning 
of images (col. 4, lines 40-42). Mojsilovic teaches the claimed, identifying one or more 
predetermined feature types within each image, or a region thereof, of a plurality of 
images of a training dataset (Fig. 9, col. 13, lines 64-67), Mojsilovic does not explicitly 
teach visual object category. However, Jain teaches the claimed, the method of 
transforming a visual object category into a model comprising parameters which define 
said visual object category (Fig. 2, col. 9, lines 30-37). Thus, it would have been 
obvious to one of ordinary skill in the data processing art at the time of the invention, to 
have combined the teachings of the cited references because Jain's teachings would 
have allowed Mojsilovic's method to provide a way to compare images in different 
schemas and to compare images faster (col. 3, lines 55-57). Mojsilovic teaches the 
claimed, comparing a set of images identified during said database search with said 
stored model and calculating a likelihood value relating to each image based on its 
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correspondence with said model (Fig. 2, col. 8, lines 19-27). Mojsilovic teaches the 
claimed, classifying said features in terms of descriptive variables defining one or more 
characteristics of said features and a spatial relationship there between (col. 2, 
lines 53-57). Mojsilovic and Jain do not teach using probability function. However, 
Essafi teaches the claimed, estimating model parameters are modeled by probability 
density functions (Fig. 8A-B, 9A-B, col. 7, lines 56-59). Thus, it would have been 
obvious to one of ordinary skill in the data processing art at the time of the invention, to 
have combined the teachings of the cited references because Essafi's teachings would 
have allowed Mojsilovic's method to provide a way without splitting the image in the 
same domain or ranges (col. 3, lines 5-8). 

The other limitations of claim 28, calculating a likelihood value relating to each 
image based on its correspondence with said model by comparing said set of images 
with said model (Fig. 2, col. 8, lines 19-27); and 

Mojsilovic do not explicitly teach ranking images. However, Jain teaches the claimed, 
ranking said images in order of said respective likelihood values, to thereby determine 
the relevance of a set of images relative to a specified visual object category (Fig. 2, 
col. 9, lines 60-63). 

13. As per dependent claim 20, Mojsilovic and Jain combined teach the claim 19. 
Jain teaches the claimed, storing said model (Fig. 2, col. 9, lines 34-37). 
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14. As per dependent claim 21 , Mojsilovic teaches the claimed, each feature is 
represented by one or more parameters, which parameters include its appearance 
and/or geometry, its scale relative to the model, and its occlusion probability (col. 2, 
lines 53-57). 

1 5. As per dependent claim 22, Mojsilovic and Jain do not explicitly teach estimating 
the probability. However, Essafi teaches the claimed, the step of comparing an image 
with said model includes identifying features of the image and estimating the probability 
densities of said parameters of those features to determine a maximum likelihood 
description of said image (Fig. 8A-B, 9A-B, col. 7, lines 56-59). Thus, it would have 
been obvious to one of ordinary skill in the data processing art at the time of the 
invention, to have combined the teachings of the cited references because Essafi's 
teachings would have allowed Mojsilovic's method to provide a way without splitting the 
image in the same domain or ranges (col. 3, lines 5-8). 

16. As per dependent claim 23, Mojsilovic and Jain do not teach using Gaussian 
probability function. However, Essafi teaches the claimed, probability density functions 
comprise Gaussian probability functions (col. 10, lines 63-65). Thus, it would have been 
obvious to one of ordinary skill in the data processing art at the time of the invention, to 
have combined the teachings of the cited references because Essafi's teachings would 
have allowed Mojsilovic's method to provide a way without splitting the image in the 
same domain or ranges (col. 3, lines 5-8). 
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1 7. As per dependent claim 24, Mojsilovic teaches the claimed, selecting the training 
dataset of images from a large dataset (database) (Fig. 1 , col. 7, lines 26-30). 

18. As per dependent claim 25, Mojsilovic teaches the claimed, at least two different 
models are created in respect of a set of images retrieved from said database (Fig. 8, 
col. 9, lines 41-43). 

1 9. As per dependent claim 26, Mojsilovic and Jain combined teach the claim 1 . Jain 
teaches the claimed, selecting one of said at least two models as said visual object 
category model (Fig. 2, col. 9, lines 60-63). 

20. As per dependent claim 27, Mojsilovic and Jain combined teaches claim 1 . Jain 
teaches the claimed, selecting step is performed by calculating a differential ranking 
measure in respect of each model, and selecting the model having the largest 
differential ranking measure (Fig. 2, col. 9, lines 60-63). 

21 . As per dependent claim 29-30, Mojsilovic and Jain combined teach the claim 28. 
Jain teaches the claimed, set of images is retrieved by means of a database on said 
specified visual object category and the input as a word or a set of words describing 
said visual object category (Fig. 2, col. 9, lines 60-63). 
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22. As per dependent claim 31 , Mojsilovic and Jain do not explicitly teach estimating 
the probability. However, Essafi teaches the claimed, the step of comparing an image 
with said model includes identifying features of the image and estimating the probability 
densities of said parameters of those features to determine a maximum likelihood 
description of said image (Fig. 8A-B, 9A-B, col. 7, lines 56-59). Thus, it would have 
been obvious to one of ordinary skill in the data processing art at the time of the 
invention, to have combined the teachings of the cited references because Essafi's 
teachings would have allowed Mojsilovic's method to provide a way without splitting the 
image in the same domain or ranges (col. 3, lines 5-8). 

23. As per dependent claim 32, Mojsilovic teaches the claimed, comparing a set of 
images retrieved from said database with said stored model and calculating a likelihood 
value relating to each image based on its correspondence with said model (Fig. 2, 

col. 8, lines 19-27). 

24. As per dependent claim 33, Mojsilovic and Jain combined teach the claim 1 . Jain 
teaches the claimed, ranking said images in order of said respective likelihood values or 
retrieving further images corresponding to said specified visual object category (Fig. 2, 
col. 9, lines 60-63). 
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25. As per dependent claim 34, Mojsilovic and Jain combined teach the claim 28. 
Jain teaches the claimed, features comprises at least two types of parts of an object 
(Fig. 2, col. 9, lines 30-37). 

26. As per dependent claim 35, Mojsilovic teaches the claimed, categories include 
pixel patches, curve segments, corners and texture (Fig. 15A-B, col. 23, lines 30-33). 

27. As per dependent claim 36, Mojsilovic teaches the claimed, all of the images of 
said set of images are used to create the model (Fig. 8, col. 9, lines 41-43). 

28. As per dependent claim 37, Mojsilovic teaches the claimed, selecting a sub-set of 
said set of images for use in creating said model (Fig. 8, col. 9, lines 41-43). 

29. As per independent claim 39, Apparatus for ranking, according to relevance, 
images of a set of images retrieved from a database relative to a specified visual object 
category, the being arranged and configured to a visual object category into a model 
defining features of said visual object category and a spatial relationship therebetween, 
store said model, compare a set of images identified during said database search with 
said stored model and calculate a likelihood value relating to each image based on its 
correspondence with said model, and to said images in order of said respective 
likelihood values. This claim is rejected under the same rationale as claim 1. 
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Response to Arguments 

30. Applicant's arguments filed 5/1/2007, 1 1/9/2007 and during interview arguments 
have been fully considered but they are not persuasive and details as follows: 

a) Applicants new drawings of Fig. 1 and 2 filed on 5/1/2007 overcomes the 
drawings objection. Therefore, the objection is withdrawn. 

b) Applicants Specification amendment filed on 5/1/2007 overcomes the 
Specification objection. Therefore, the objection is withdrawn. 

c) Applicants cancelled old claims 1-18 and added new claims 19-39, filed on 
5/1/2007 over comes the claims objection. Therefore, the objection is withdrawn. 

d) Applicants argument regarding claims 1 , 17-18 were rejected under 35 
U.S.C. 112, second paragraph for indefinite because the claims were distinct 
from the specification and claims 5 and 8 were rejected for using the indefinite 
phrase "and/or" respectively. However, Applicants cancelled old claims 1-18 and 
added new claims 19-39, filed on 5/1/2007, and moot to discuss the rejection. 

e) Applicants argument regarding claims 1-18 rejection under 35 U.S.C. 101 
because independent claims 1,17-18 were directed to non-functional descriptive 
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material, i.e., abstract ideas. However, Applicants cancelled old claims 1-18 and 
added new claims 19-39, filed on 5/1/2007, and moot to discuss the rejection, 
f) In response to applicant's argument that there is no suggestion to combine 
the references, the examiner recognizes that obviousness can only be 
established by combining or modifying the teachings of the prior art to produce 
the claimed invention where there is some teaching, suggestion, or motivation to 
do so found either in the references themselves or in the knowledge generally 
available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1 071 , 5 
USPQ2d 1596 (Fed. Cir. 1988)and In re Jones, 958 F.2d 347, 21 USPQ2d 1941 
(Fed. Cir. 1992). In this case, Jain's abstract states "Object attributes can be 
extracted over the entire image or over only a portion of the object. A schema is 
defined as a specific collection of primitives." Col. 16, lines 54-57 state that 
"Structure ~ This primitive is used to capture the shapes which appear in the 
image" Primitives include "Global Color", "Local Color", "Structure", and 
"Texture". See also, col. 7, lines 60-66, states that "Primitive are either global, 
computer over an entire image, or local, computer over smaller regions of the 
image." So shapes (objects) within an image appear to be disclosed. In this 
sense, it appears that Jain would be able to disclose that the object (lion on one 
side) would be able to say the image is the same an image of a lion on the other 
side of the image, if the image shapes were extracted and the localization (which 
I cannot find in the claim), however the "over the entire image" or over "local" can 
be interpreted as localized determination of an object" as argued. Further, Jain 
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teaches as "the new images are provided as inputs to the Visual information 
retrieval (VIR) Engine 120. Visual objects associated with a particular domain 
may be stored in a portion or a partition of the entire database. Visual objects of 
multiple domain or subsets of a domain could be stored in separate databases or 
nay be stored in one database. The VIR Engine 120 has two modules, image 
analysis module 122 and image comparison module 124. The image analysis 
module 122 receives inputs form either modules 108 or 1 10 are to generate a 
query target or from the insertion module 1 12 for adding the new image into the 
database 112. This is the classification of visual image objects part. Then the 
second module, image comparison module 124 receives a query target feature 
vector (FV) and a FV for the image being tested or compared one at a time to 
the query from the database engine 130. For details see Jain Fig. 2, col. 3, 
lines 30-65). Mojsilovic teaches the claimed limitation, "identifying one or more 
predetermined feature types within each image or a region, a plurality of images 
of a training dataset" as the image similarity metric embodies the experimental 
results and may used to annotate images or to search the database 104, see 
Mojsilovic at Fig. 9, lines 64-67. 

Conclusion 



31 . THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 



Application/Control Number: 10/813,201 Page 14 

Art Unit: 2164 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Sathyanarayan Pannala whose telephone number is 
(571 ) 272-41 1 5. The examiner can normally be reached on 8:00 am - 5:00 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Charles Rones can be reached on (571) 272-4085. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Sathyanarayan Pannala/ 
Primary Examiner 
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May 13, 2008 



